Compact Representation of Value Function in Partially Observable Stochastic Games

Value methods for solving stochastic games with partial observability model the uncertainty about states of the game as a probability distribution over possible states. The dimension of this belief space is the number of states. For many practical problems, for example in security, there are exponentially many possible states which causes an insufficient scalability of algorithms for real-world problems. To this end, we propose an abstraction technique that addresses this issue of the curse of dimensionality by projecting high-dimensional beliefs to characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our two main contributions are (1) novel compact representation of the uncertainty in partially observable stochastic games and (2) novel algorithm based on this compact representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm over the compact representation dramatically increases the scalability compared to the state of the art

Empirical Game-Theoretic Methods for Strategy Design and Analysis in Complex Games.

Complex multi-agent systems often are not amenable to standard game-theoretic analysis. I study methods for strategic reasoning that scale to more complex interactions, drawing on computational and empirical techniques. Several recent studies have applied simulation to estimate game models, using a methodology known as empirical game-theoretic analysis. I report a successful application of this methodology to the Trading Agent Competition Supply Chain Management game. Game theory has previously played little—if any—role in analyzing this scenario, or others like it. In the rest of the thesis, I perform broader evaluations of empirical game analysis methods using a novel experimental framework. I introduce meta-games to model situations where players make strategy choices based on estimated game models. Each player chooses a meta-strategy, which is a general method for strategy selection that can be applied to a class of games. These meta-strategies can be used to select strategies based on empirical models, such as an estimated payoff matrix. I investigate candidate meta-strategies experimentally, testing them across different classes of games and observation models to identify general performance patterns. For example, I show that the strategy choices made using a naive equilibrium model quickly degrade in quality as observation noise is introduced. I analyze three families of meta-strategies that predict distributions of play, each interpolating between uninformed and naive equilibrium predictions using a single parameter. These strategy spaces improve on the naive method, capturing (to some degree) the effects of observation uncertainty. Of these candidates, I identify logit equilibrium as the champion, supported by considerable evidence that its predictions generalize across many contexts. I also evaluate exploration policies for directing game simulations on two tasks: equilibrium confirmation and strategy selection. Policies based on computing best responses are able to exploit a variety of structural properties to confirm equilibria with limited payoff evidence. A novel policy I propose—subgame best-response dynamics—improves previous methods for this task by confirming mixed equilibria in addition to pure equilibria. I apply meta-strategy analysis to show that these exploration policies can improve the strategy selections of logit equilibrium.Ph.D.Computer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/61590/1/ckiekint_1.pd

Performance Comparison and Implementation of Bayesian Variants for Network Intrusion Detection

Bayesian classifiers perform well when each of the features is completely independent of the other which is not always valid in real world applications. The aim of this study is to implement and compare the performances of each variant of the Bayesian classifier (Multinomial, Bernoulli, and Gaussian) on anomaly detection in network intrusion, and to investigate whether there is any association between each variant’s assumption and their performance. Our investigation showed that each variant of the Bayesian algorithm blindly follows its assumption regardless of feature property, and that the assumption is the single most important factor that influences their accuracy. Experimental results show that Bernoulli has accuracy of 69.9% test (71% train), Multinomial has accuracy of 31.2% test (31.2% train), while Gaussian has an accuracy of 81.69% test (82.84% train). Going deeper, we investigated and found that each Naïve Bayes variants performances and accuracy is largely due to each classifier assumption, Gaussian classifier performed best on anomaly detection due to its assumption that features follow normal distributions which are continuous, while multinomial classifier have a dismal performance as it simply assumes discreet and multinomial distribution

Honeypot Allocation for Cyber Deception in Dynamic Tactical Networks: A Game Theoretic Approach

Honeypots play a crucial role in implementing various cyber deception techniques as they possess the capability to divert attackers away from valuable assets. Careful strategic placement of honeypots in networks should consider not only network aspects but also attackers' preferences. The allocation of honeypots in tactical networks under network mobility is of great interest. To achieve this objective, we present a game-theoretic approach that generates optimal honeypot allocation strategies within an attack/defense scenario. Our proposed approach takes into consideration the changes in network connectivity. In particular, we introduce a two-player dynamic game model that explicitly incorporates the future state evolution resulting from changes in network connectivity. The defender's objective is twofold: to maximize the likelihood of the attacker hitting a honeypot and to minimize the cost associated with deception and reconfiguration due to changes in network topology. We present an iterative algorithm to find Nash equilibrium strategies and analyze the scalability of the algorithm. Finally, we validate our approach and present numerical results based on simulations, demonstrating that our game model successfully enhances network security. Additionally, we have proposed additional enhancements to improve the scalability of the proposed approach.Comment: This paper accepted in 14th International Conference on Decision and Game Theory for Security, GameSec 202

A Polynomial Time Algorithm for Spatio-Temporal Security Games

An ever-important issue is protecting infrastructure and other valuable targets from a range of threats from vandalism to theft to piracy to terrorism. The "defender" can rarely afford the needed resources for a 100% protection. Thus, the key question is, how to provide the best protection using the limited available resources. We study a practically important class of security games that is played out in space and time, with targets and "patrols" moving on a real line. A central open question here is whether the Nash equilibrium (i.e., the minimax strategy of the defender) can be computed in polynomial time. We resolve this question in the affirmative. Our algorithm runs in time polynomial in the input size, and only polylogarithmic in the number of possible patrol locations (M). Further, we provide a continuous extension in which patrol locations can take arbitrary real values. Prior work obtained polynomial-time algorithms only under a substantial assumption, e.g., a constant number of rounds. Further, all these algorithms have running times polynomial in M, which can be very large

Approximation methods for infinite bayesian stackelberg games: Modeling distributional payoff uncertainty.

ABSTRACT Game theory is fast becoming a vital tool for reasoning about complex real-world security problems, including critical infrastructure protection. The game models for these applications are constructed using expert analysis and historical data to estimate the values of key parameters, including the preferences and capabilities of terrorists. In many cases, it would be natural to represent uncertainty over these parameters using continuous distributions (such as uniform intervals or Gaussians). However, existing solution algorithms are limited to considering a small, finite number of possible attacker types with different payoffs. We introduce a general model of infinite Bayesian Stackelberg security games that allows payoffs to be represented using continuous payoff distributions. We then develop several techniques for finding approximate solutions for this class of games, and show empirically that our methods offer dramatic improvements over the current state of the art, providing new ways to improve the robustness of security game models

Imitative Follower Deception in Stackelberg Games

Information uncertainty is one of the major challenges facing applications of game theory. In the context of Stackelberg games, various approaches have been proposed to deal with the leader's incomplete knowledge about the follower's payoffs, typically by gathering information from the leader's interaction with the follower. Unfortunately, these approaches rely crucially on the assumption that the follower will not strategically exploit this information asymmetry, i.e., the follower behaves truthfully during the interaction according to their actual payoffs. As we show in this paper, the follower may have strong incentives to deceitfully imitate the behavior of a different follower type and, in doing this, benefit significantly from inducing the leader into choosing a highly suboptimal strategy. This raises a fundamental question: how to design a leader strategy in the presence of a deceitful follower? To answer this question, we put forward a basic model of Stackelberg games with (imitative) follower deception and show that the leader is indeed able to reduce the loss due to follower deception with carefully designed policies. We then provide a systematic study of the problem of computing the optimal leader policy and draw a relatively complete picture of the complexity landscape; essentially matching positive and negative complexity results are provided for natural variants of the model. Our intractability results are in sharp contrast to the situation with no deception, where the leader's optimal strategy can be computed in polynomial time, and thus illustrate the intrinsic difficulty of handling follower deception. Through simulations we also examine the benefit of considering follower deception in randomly generated games

Concealing Cyber-Decoys using Two-Sided Feature Deception Games

An increasingly important tool for securing computer networks is the use of deceptive decoy objects (e.g., fake hosts, accounts, or files) to detect, confuse, and distract attackers. One of the well-known challenges in using decoys is that it can be difficult to design effective decoys that are hard to distinguish from real objects, especially against sophisticated attackers who may be aware of the use of decoys. A key issue is that both real and decoy objects may have observable features that may give the attacker the ability to distinguish one from the other. However, a defender deploying decoys may be able to modify some features of either the real or decoy objects (at some cost) making the decoys more effective. We present a game-theoretic model of two-sided deception that models this scenario. We present an empirical analysis of this model to show strategies for effectively concealing decoys, as well as some limitations of decoys for cyber security